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ABSTRACT 

Three studies were conducted to investigate the 
development of the ability of individuals of varying ages to use 
macrorules for paraphrasing expository text. Macrorules were defined 
as the general rules of deletion, superordination, selection, and 
invention that underlie comprehension of prose. In the first study, 
18 fifth grade, 16 seventh grade, 13 tenth grade, and 20 college 
students were given f^xpository texts and told to summarize them. They 
were allowed to do anything that would help them write good 
summaries, including taking notes, underlining text, writing rough 
drafts, alid keeping their notes and rough drafts at hand while 
summarizing. The results were marked for use of macrorules by 
independent raters,. In the second study, college rhetoric instructors 
( ''experts" ) completed a similar task. In addition to summarizing the 
material, they were asked to talk about their methods for completing 
the task and about how they taught their students to summarize. In 
the third stndy, 21 junior college students C^novices" ) , completed a 
similar tasH. Results showed that older high school students, college 
students, and "experts" -wrote better and used rules more efficiently 
while writing than did younger students and novice writers. There was 
also a marked tendency on the part of more mature students to 
rearrange material across paragraphs, combining according to common 
topic. (FL) 
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Macrorules for Summarizing Tests: 
The Development of Expertise 

The ability to summarize Information lis an Important study skill 
Involving both comprehension of, and attention to, Importance at the* 
expense of trivia. Recent: evidence suggests that this may be a late 
developing skill. When wrl^.lng summaries, college and older high school 
students outperform younger children In their propensity to plan ahead. In 
their sensitivity to fine gradations of Importance In the text, and In 
their ability to condense mora Ideas Into the same number of words (Brown, 
Day, & Jones, In press). The ability to recursively work on Information to 
render It as succinctly as possible requires judgment and effort, knowledge 
and strategies. 

When chfldren are .asked to .summarize age--approprlate material, they 
are able to employ simple deletion procedures at a relatively early age. 
For example, Johnson (1978, in press) asked grade school and college 
students to orally summarize well-formed stories, ^The standard strategy of 

the children was deletion. But children as young as first grade did use 

\ 

some transformational condensation rules; approximately 30Z of the summary 
units produced by first, third afid fifth graders rejpresented story nodes by 
transformations of the original text content, compared with 60Z for college 
students. Using a more difficult task, writing a summary of, much longet, 
and less well-formed stories. Brown, Day, and Jones (in press) found that 
fifth graders were able to delete both trivial and redundant material but 
there was little evideilce of more complex transformational rules of 
condensation until the later high school 'years. 
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In the Brown, Day, and Jones study, fifth and seventh graders, 
re9uired to write a summary of a lengthy story, appeared to treat the task 
as one of deciding If to. Include or delete elements that actually occurred 
in the surface structure of 'the original text» Brown et^ al referred' to 
this as the copy - delete strategy ♦ In general the strategy is as follows: 
(a) read text elements sequentially; (b) decide for each element on 
inclusion or deletion; (c) if inclusion is the verdict, copy it more or 
less verbatim from the text. The same general strategy is employed by 
fifth and seventh grade notetakers (Brown & Smiley, 1978) and outliners 
(Brown, 1981) • Interviews conducted with seventh-eig^hth grade students 
concerning their study and research habits again suggest that this is a . 
common method ♦ The students often reported that they copy verbatim froa 
research sources when preparing papers; they had little appreciation of thg 
reed to extract the main points and restate them in their own words ♦ 

In contrast, the strategy of older high school and college students in 
the Brown, Day, and Jones study differed radically from the copy-delete 
ploy* They systematically departed from both the surface wording and the 
temporal sequence of the text, combining across paragraphs, rearranging by 
topic cluster and stating the gist in their own words • Ihey relied heavily 
on transformational rules to pwduie a synopsis in their own words of the 
essential meaning of the text.*' . ^ 

In this paper, we will examine the Wsic condensation rules employed 
by children and adults as they summarize expository texts rather than 
stories. But what are these rules? In the summarization model proposed by 
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v^n Dljk and Klntsch (van Dljk & Kintsch, 1977,'^ntsch & van Dijk, 1978), 
the Information to be Included In a summary Is determined by macrorules 
(processes of deletion, generalization and Integration) that operate on the 
propositions of the Input text to produce'^a macros tructure. Based on this 
analysis and an Informal consideration of summarization protocols obtained 
from children and adults, we Identified six basic rules of summarization. 

Two of t;he six rules Involve the deletion of unnecessary material. 
One should obviously delete material that is trivial , and even grade school 
children are quite adept at this if the. form and content of th^ material is 
familiar (Brown, Day, & Jones, irv press; Johnson, 1978). One should also 
delete material that, although it is important, is also redundant. Kintsch 
and van Dijk's system also includes these two deletion rules. Two of the 
rules of summarization involve the substitution of a superordlnate term or 
event for a list of items or actions. For example, if a text contains a 
list such as: cats, dogs, goldfish, gerbils and parrots, one can 
substitute the term pecs. This is Kintsch and van Dijk's generalization 
rule. Similaijly, one can substitute a superordlnate action for a list of 
subcomponents of that action, I.e., John went to London, for: John left 
the house, John went to the train station, John bought a ticket, etc., etc. 
This Is roughly comparable to Kintsch and van Dljk's (1978) integration 
rule. The two remaining rules have to do with providing a summary of a 
main constituent unit of text, the paragraph. The first rule is — select 
a topic sentence, if any, for this is the author's summary of the 
paragraph. The final rule is — • if there is no topic sentence, invent your 



O 
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own* These operations are roughly equivalent to Klntsch and van Dljk's 
construction rule* 

These basic rules seem to capture the essence of the methods of 
condensation actually used by students when engaged In the formal task of 

summarizing, they also seem t;o be the rules used by more mature high -school 

II 

students when notetaklng and outlining (Brown, 1981; Brown & Smiley,^ 1978) . 
Klntsch and van Dljk argue chat these raacrorules of deletion, 
superordlnatlon, selection and Invention are general rules underlying 
comprehension of texts, not just specific rules for carrying out a ,>ummary 
writing task. 

Three studies are reported here. In the first study, we examine the 
developmental trend associated with the use of macrorules when paraphrasing 
expository^ texts. In the second study, we examined experts" use of 
summarization rules using on-line "talk aloud" protocols. Following our 
consideration of experts we turned oar attention to novices; in the third 
study we examined the potential diagnostic power of our developmental norms 
by considering the performance of junior college students, a population 
known to experience problems in critical reading and effective studying* 

EXPERIMENT 1 

Method 

Subjects . The subjects were 18 fifth graders, 16 seyenth graders, 13 
tenth graders and 20 college students. Their mean ages were 10.7, 13.11, 
15,4 and 18.1 respectively. To the best of our knowledge they were 
experimentally naive. The fifth, seventh, and tenth graders were from 
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rural Central Illinois. According to their teachers, they had no 

> 

discernible reading problems and they were not receiving any extra help 

5 * 

with reading or study skills. The college students were fresroan 



Unlvferslty of Illinois undergraduates enrolled in an introductory 



psychology class. 



Materials * Two expository texts were constructed for use in 



Experiments 1-3. We selected, modified and rewrote suitable seven* grade 
g^^raphy texts to serve the purposes of this study. One text, entitled 
"Desert," was about how plants and animals survive the harsh desert 



climate. The other text, "Noise," was about the adverse effects that noise 



can have on one's health and hearing. Both texts were rewritten so that ' 
they were of approximately equal length (492 and 532 words, 36 and 42 



lines), comparable readability level (Dale-Chall readability scores of 5.29 

0 

and 5.32), and of approximately the same number of idea units (81 and 68, 



as determined by 15 college student raters). The idea units were rated in 



terms of their structural importance to the text by 11 additional college 



students • 



All texts were constructed so that five of the six rules could be 



used. Across texts, the number of segments that would elicit each type of 
rule was held roughly constant. Each rule could be applied at least three 
but never more than five times on any given text. The five rules (with van 



Dijk and Kintsch [1977] terms in parentheses) were (1) deletion (deletion) 
of unimportant or trivial information, (2) deletion (deletion) of redundant 



information , (3) superordination (generalization) of lists, i.e.. 
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substitution of category name for Instances pf a category, (4) selection 
of a topic sentence, l»e», near verbatim use of a topic sentence from the 
text and (5) Invention (construction), l.e», creation and use of a topic 
sentence that did not appear In the text but easily could have. 

The first dele^.lon rule was to eliminate unimportant information from 
the summary. To i^ncourage^he use of this rule, the texts were written so 
that they contained minor details about the topics, details that 
independent college students rated as ujilmportant. Importance was defined 
as those units receiving a rating of 3 or 4 on a four point scale. The 
second deletion rule wa$ tp eliminate redundancy. Redundant information 
was Included In the texts by rewording and then restating some of the more 

c 

Important sentences. All redundant information received a 3 or 4 
importance-level rating. Therefore, unimportant and redundant information 
did not overlap. 

The texts were written so that the superordination of list^ rule could 
be applied three times. Each text contained lists pf category ipembers 
whose superordinates were familar to grade school children. For example, 
in the desert text, flowers would be an appropriate superordinate for the 
list of exemplars.: "daisies, poppies, marigolds and lilies." 

Finally, in order to make the selection and invention rules generally 

applicable across texts, paragraphs were written around and in support of a 

topic sentence. College students rated all topic sentences as highly 
* 

Important. For cases where selection was appropriate, the topic sentence 
wad left in the text and read by the subject. For invention, the topic 
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sentence was deleted, and minor stylistic changes werg mad§- to the text to 
make it read smoothly. 

/ 

A final pilot study was run to find out if subjects would use the 
topic sentence rule for each paragraph "^if it appeared explicitly in the^ 
texts. A version of the texts that contained all the topic sentence's was 
giyen to groups of undergraduates to summarize. Their summaries were ji^st 
as likely to include topic sentences on those paragraphs targeted for 
^^selection as on those targeted for invention tests, suggesting that the 
paragraphs themselves were similar with the exception of the main 
manipulation, presence or absence of an explicit topic sentence. 

During and after the calibration of the texts, sections were rewritten 
to ensure normal discourse coh^^ion and flow, ^^hen the texts were finally 
rated and calibrated it was possible to .predict where each of the five 
rules should be used and the dependent measure ^was the number of times a 
rule was used given that it was appropriate. In summary, the texts were 
constructed specifically to elicit each of the rules of summarization. 
Furthermore, reading difficulty was held constant, the frequency of 
occurrence of each rule type was controlled, and the appropriate rule could 
be ideotified in advance. 

Procedure. Fifth, seventh, and tenth graders were tested as a class 
in two forty-minute sessions. The college students were also, tested as 
groups but in one one-hour period. Half of the subjects within each age ^ 
group read "Noise*' first and half read ''Desert" first. Subjects were given 
a text and asked to read it three times. After reading, they were asked to 

X 

\ 

■ -id \ • , 



'.^ / , Development of Expertise 



write what they thought was a good summary of the text. When they had 
completed theit f.irst summary, they were asked to put it aside and to write 
a 60 word summary; this was selected because it was the approximate length 
takeh, by a group of experts when asked to^rovide a brief but coherent 

summsiry of these texts. Subjects were told to do anything that would help 

r 

them write good summaries. They could take notes, underline the text, 

D ^ 

write rough drafts, And keep the text and their'^notes in front of them. 

Hdwever, the^ were not allowed to use their unconstrained summaries when 

writing the 60 word summaries* " At. the end of the session, all the 

materials were collected. The procedure was repeated^ in the second session 

using the text ?>ot previously summarized. 

* * 

The summaries were corrected for spelling and punctuation ,ai?^ then 
typed onto Index cards {so ^that information concerning age and conditlN)n 
would not be available the raters. They were then scored by two* 

1 

independent raters, with^an inter-rater reliability of .96. 

Resul^ and discussion . There were five summarization rules tijiat 
could be employed. Because of wide variability with age in the use of 
these rules separate analysis of variance were conducted on each rule type. 
Stories were tred^ed as a fixe4 effect, as the artificial construction of 
these stories was^su^that generalization ^o. the class of naturally 
occurring stories was noKthought reasonable; these stories were designed 
to be most likely to elicit^ the str^ gies under consideration. 
Occasionally a main effect of story was fopnd. * This 'Effect was always due 
to t|ie "Noise" text being more difficult than the "Desert" text. As the 
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effect of stories did not enter Into any Interactions, the data were 
J cojnbined across stories for purposes ^of the analyses. 

In addition, nhere was rarely, an effect found for the second variable, 
coKs'tralhed or unconstrained summary. The exception will be noted In the 
text. The- unconstrained summaries were longer tfian the constrained 
summaries at all ages, and at all ages the students obeyed the length' 
restriction of the- constrained summaries. 

All of the analyses of variance were mixed, with Age (grades 5, 7, 10, 
and college) as the between subjects variable and Story (Noise/Desert;) and 
Summary Type (Const^^ned/Uitconstralned) as within subject variables. All 
^jr^ analyses were conducted on the arc sine tranformed mean proportion of 
ocfcurrences of rule use.* ' * 
• • Both of the deletion rules,, delete trivia x>r delete redundancy, were 
used effectively by all age groups (see Table 1). An analysis of variance 
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reveaiUd no significant effects.* Performance was consistently In the 90% 

range, or better. Subjects. as young fifth grade are able to delete both 

trivial information (replicating Brown, Day, Jon^s, In press) and J 

redundant material. Even though., the, redundant material wds Important to 

the theme, fifth graders can omit It from their summaries. This Is an - 

*» * 

Important finding for It confirms that the- younger children In th*s ^tudy 
w^^^jfi^ble to employ at least some of the rules of summary,, and were not 
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just experimental foils used to provide a baseline against which 
Improvement with age could be measured. 

Consider next the superordlnatlon rule. Faced with text segments 
where this rule can be applied, there are four options open to the, subject: 
(1) delete the unit entirely, (2) repeat It exactly, (3) use a 
superordlnate inefficiently, and (4) use\a superordlnate efficiently. For 
example, consider a unit of the ''Desert" t^t: "Daisies, poppies, 
marigolds, and lilies stay in the form of see^s." The unit could be 
deleted because that unit of text will not be featured in the summary (1). 
If it were included at all it can be repeated verbatim (2) or an attempt to 
use a superordlnate can be made. Efficient superordlnatlon (4) would be - 
when the superordlnate "desert flowers," "flowers," or "annual flowers" is 
substituted, for the subordinate list. Inefficient superordlnate also 
occurred, where the subject included some of the subordinates with the 
superordlnate, thereby failing to gain the full advantage of using the 
strategy (e.g.. Flowers: poppies, and lilies stay in the form of seeds). 

As can be seen in Table 2, oldet subjects are more likely to produce 
efficient superordinates on those occasions when they do not delete the 
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entire unit. To test this we computed for each subject the conditional 
probability of producing a good superordlnatlon given that the segment of 
text was not deleted. A mixed analys-^s of variance with Age (3) and 
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Stories (2) as between subjects variables and Summary Type as a within 
subjects variable was conducted on these conditional probabilities (arc 
sine transformation used)* Only three ages were included in the formal 
analysis because the fifth graders deleted so many of the superordination 
units. The analysis revealed a main effect of Age, F(2,41) « 12.76, £ < 
♦001 and of Summary Type, F(l,41) « 29.70, £ < ,001. All subjects used the 
superordination rule more efficiently under space pressure thaa when 
unconstrained by a word limit, and the probability of using the rule 
effectively increased with age. Post-hoc tests revealed that the age 
difference was carried by the seventh graders performing less well than the 
older subjects. Seventh graders tended to repeat (.33) or use the rule 
inefficiently (.20) rather than efficiently when unconstrained by space 
pressure* Even under constrained conditions approximately half of the 
seventh graders' responses are repetitious and poor (.30) rather than good 
(♦31). By contrast, tenth graders and college students rarely repeat (.04) 
or use the rule inefficiently (.06) when constrained by a word limitation. 

Age differences in the use of the selection rule were also apparent. 
The selection data are shown in Table 3* The main effects of Age, 

t 

Insert Table 3 About Here 



F(3,67) = 14.43, £ < .001, Summary Type, P(l,67) = 9.59, £ < .002, and ^ 
Story, F(l,67) = 20.79, £ < .001 were all reliable, as was the Age x 
Summary Type interaction, F(3,67) » 2.82, £ < .05. Use of the selection 
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nile Increased with age In both conditions. There were no differences 
between conditions for the younger groupsT~howev^^"~w 
decreased their use of the selection rulfe when constrained by a word 
limitation. One explanation for this finding Is that mature summarlzers, 
when pressed for space, drop the selection rule which Is somewhat space 
consuming, and substitute a more oblique form of reduction, similar to 
Invention, i.e., they combined across paragraphs and expressed the 
essential gist of large bodies of text In few words. Therefore, they did 
not receive a score for using the available topic sentences of several 
paragraphs. T^ils is a common strategy of expert summarlzers (see 
^ Experiment 2)- 

The final rule to be considered is that of invention . The mean 
proportion of invention rule use is also presented in Table 3. Analysis of 
variance resulted in a main effect of Age, F(3,67) = 18.42, p < .001 but no 
other main effects or interactions were reliable. The ability to invent 
explicit topic sentences to state the implicit main idea of paragraphs is 
difficult, and develops with age. Use of the invention rule by fifth 
graders was a rare oocirrence. College students invent but only on half of 
the occasions when it would be appropriate to do so. 

In summary, even fifth graders know how to delete trivial or redundant 
elements of simple texts, but older subjects outperform younger subjects in 
the use of more complex condensation rules. When required to use a 
superordinate substitution rule, college students and tenth graders 
produced good superordinates, but younger children use the superordinate 
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rules less frequently, and when they do attempt to use the rule they often 
use It inefficiently. The use of selection gradually Increases with age as 
does-ltivenMon. TheJ.ayjeatliKLjrule Is the most difficult, with very little 
use of the rule made by fifth and seventh graders. Tenth graders use the 
rule 'on one-third of appropriate occasions and even college students ^use 
the rule only on half of the units where It would be appropriate. 

Given that even college students demonstrated considerable room for 
Improvement, particularly In their use of the Invention rule, we decided 
next to examine the efficiency of "experts'* In applying the five basic ^ 
rules of summary. 

\, EXPERIMENT 2 

Methods 

Subjects . We contacted six fourth year graduate students In the 
English Department at the Unlve^rslty of Illinois who had taught freshman 
rhetofic courses at least twice. From that sample, we selected two 
cooperative subjects who were able to comply with the talk -aloud procedure 
while attempting to summarize and who performed well on an initial test of 
summarization skills. Note that these subjects, in addition to their 
greater experience, were more highly selected than the undergraduates who 
took part In Experiment 1 (see Experiment 3 for a discussion of the samples" 
included in these studies). 

Procedure. The experts worked on the same passage used in Experiment 
1. the first passage, the procedure was identical to Experiment 1, 

with subjects writing both a constrained (60 words) and an unconstrained 
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sununary of the text. Two weeks after completing the 'standard assignment we 
presented the second text (text order counterbalanced) and again asked the 
subjects l-o prepare an unconstrained version followed by a 60 word summary. 
However, we preceded this second session by asking subjects about how they 
taught summarization skills to their students and what they thought were 
the basic rules of a good summary. In addition, during their actual 
attempt to provide a summary, we asked them to "talk-aloud*' while working. 
They were asked to try to tell us what they were doing; they were told to 
tell us anything that came to mind, no matter how trivial, and to describe 
the processes they went through as they worked. We asked them to reflect 
on what they were doing and to describe any general rules they were 
conscious of using. These protocols were tape recorded and transcribed. 
Resu^lts and discussion . Consider first the rule use data comparable 

to that gathered from the students-^txuJExpLarimeniL.-L*. As„expected, 

performance on the deletion rules was almost perfect, and no further 
consideration was given to these data/ The experts' data on the remaining 
rules are presented in Figure 1, together with the comparable data from 
first year undergraduates (from Experiment 1) and first year students from 
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junior college (from Experiment's) . The expeLus uaed the superordination 
rule perfectly compared to the 70% level set by the four-year college 
students. There were no differences between populations in the use of the 
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selection- rule. However, the experts used the difficult invention rule 
much more than did the four-year college students (.84 vs. .49). Indeed, a 
case could be made that the experts performed perfectly because on the rare 
occasions that tHey did not receive a "correct score" for invention use, 
they had combined two paragraphs into one, therlUy losing credit for one 
' topic sentence use. This strategy of combining across paragraphs was also 
largely responsible for the somewhat low performance on the selection 
strategy. Combining tvro paragraphs and using one topic .sentence for both 
depressed scores on the selection rule, an obvibus limitation to the 
scoring system that had not been a problem when considering the protocols, 
of the less experienced stxidents. Rarely did any of the high school 
students combine paragraphs. Experts, bowevfer, favored the paragraph 
combining strategy and attempted to use it whenever possible. 

Consider now the verbal protdcols. in the open-ended interviews prior 
to actually summarizing, the experts showed a surprising lack of evidence 
that they knew any effective rules for summarization. Their description of 
what a good summary was, and what to t^ll students, was essentially similar 
to that contained In rhetoric text books (Bessey & Coffin, 1934). They 
stressed that a summary is a concise statement of the theme and that one 
should avoid unnecessary repetition, be concise, include only main ideas, 
etc. , but there was no mention of a systematic set of rules for 
accomplishing this end. 
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Dur ing th eir attempt to summarize, however, the experts made frequent 

mention of the basic rules* The protocols were lonp and discursive. Forty 
percent of the comments were judged to be a statement of a rule, 14Z were 
judged to be irrelevant and ^5% of the discourse focused on passage v 
content* Of the statements judged to be a reference to rule use, 68% were 
an explicit statement of one of the five rules. Examples of verbatim 
statements are given in Table 4. ' « 
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These experts were unable or unwilling to give a precise statement of 
the rules that might be used prior to attempting to summa^rize a text. They 
spoke in very general terms about finding "main ideas" and "being concise," 
etc* As Ericsson and Simon (1980) point out, although verbal reports can 
provide invaluable data concerning human cognitive processing, the least 
likely procedure for obtaining accurate verbal descriptions is where 
subjects are asked to report retrospective ly about how they might act 
generally in imaginary situations -(see a'.so Brown, Bransford, Ferrara, & 
Campione, in press, for a discussion of this, pblnt). In confirmation, the 
experts here were less than informative when asked to talk in general terms 
about the processed of summarization. In contrast, however, in the 
concurrent verbalizations, produced when they were faced with the task of 
summarizing a passsfge, they were much more explicit about the rules they 
were employing* Again, as Ericsson and Simon point out, requiring on-line 

H 
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reporting of <tppcific cognitive processes that the subject is actually 
using is a more optimal procedure for eliciting reliable and informative 



verbal reports. Under these pocedures the experts reported the use of 
specific rules for summarizing texts; and, for the most part, the rules 
they described vzere the five basic rules of deletion, superordination and 
topic sentence manipulation. 

In addition, it was observed that the experts' general procedure 
differed sharply from that of the younger children in Experiment 1 who went 
through the text sequentially deleting or copying segments. Experts 
accorded special status to the topic sentence, selecting or inventing them 
first and then writing their summary around and in support of the topic 
sentences. The only other dominant rule that was used by experts and 
repeatedly appeared in the protocols was the combining-paragraphs (see 
Table 4). Experts used the rule routinely. Younger subjects rarely 
attempted to combine across paragraphs, seeming instead to be "captured" by 
the paragraph structure provided in the input passage. — - 

EXPERIMENT 3 ' 

Having examined experts' summarization performance, we turn now to 
novices. In order to examine the diagnostic value of pur <age norms, we 
repeated Experiment 1 using junior college students, a population thought 
to experience difficulty employ^ basic skills of critical reading and 
studying.^ A consideration of the traditional educational research 
literature would suggest that junior college students are not alone in> 
their difficulty with the task of adequately abbreviating text; elementary 
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school children (Germane, 1921a, 1921b) and Air Force recruits (Stordahl & 

o 

Christensen, 1956) demonstrate poor summarizing skills. In fact, 
summarizing is just one of several study techniques that immature students 
fail to employ well (Anderson & Arrabruster, in press). For example, 
educators complain that high school students (Dynes, 1932; Beauchamp, 1923; 
Germane, 1921b), recruits for the armed forces (Weinstein, 1978; Stordahl & 
Christensen, 1956) and even some college undergraduates (McClusky & Dolch, 
1924) lack basic notetaking and/or outlining skills and early observations 
of high school students' study habits revealed that their notes and 
summaries tend to be written somewhat indiscriminately, with equal weight 
given to major and to minor points (Germane, 1921a; Beauchamp, 1923). An 
examination of the validity of these traditional claims, using our 



sensitive diagnosis of rule use, seemed timely. 
Methods 

Subjects . Twenty freshman students attending a Central Illinois 
junior college served as subjects. All were enrolled in an English course 
that fulfilled the freshman rhetoric requirement at that college and at 
many four-year universities. That is, students could receive credit for 
this course should they continue their education at a four--year 
institution. The students were not , therefore, diagnosed as having any 
reading or writing problems on the basis of tests administered on entry to 
^the college. In general, they were in a college preparation stream. 
En^l4^H was their first language. ^ 
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It should be pointed out, however, that given the selection of 
students entering junior college and the University of Illinois, the sample 
of students would be expected to have lower scholastic achievement. 
According to the Illinois Board of Higher Education, approximately 25% of 
college-age students enroll in four-year institutions. Given that the 
University of Illinois is one of the most prestigious in the State and has 
the highest entry requirements of all the State colleges, it can be assumed 
that the' college students taking part in Experiment 1 were at least in the 
top 2J% of the distribution and more likely at the upper end of that 25%. 
In contrast, an additional 43% of colTege-age students atten'd junior 
colleges in the State. As the junior colleges have *rio entry requirements 
beyond high school graduation. It can be assumed that the junior college 
sample of Experiment 3 would be at the middle range of the distribution of 
academic credentials. In short, the junior college students came from the 
same population as the "normal" high school students of Study 1 with the 
top 25% selected out. 

Maeerials > The materials were the s5me as those used in Experiment 1, 
Procedure , The procedure was the same as that used with the college 
students in Experiment 1, Subjects were seen for about one hour in small 
groups during which they wrote four summaries, one unconstrained and^one of 
60 words on each of two tekts. Subjects had the texts available t5 them ' 
throughout the ex^periment so they could refer back to them while writing 
their summaries. In addition, scratch paper was provided and students were 
tofd that they could take notes, write a draft or mark the text; they were 
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permitted to use any method to facilitate producing good summaries except 
using the unconstrained summary to write the 60^word version. 

Results and discussion . Summaries were typed onto index cards dnd two 
independent raters scored them for rule use. All analyses were carried. out 
on the arc sine transformed mean proportion of occurrences of rule use. 
Stories were treated as a fixed effect. ' 

Junior college students demonstrated a rudimentapr understanding of 
the summarization task by deleting trivial and redundant information. 
Junior college students eliminated 92% of the unimportant and 94% of the 
redundant material. Junior college students compared favorably to the 
four-year university students of Experiment 1, who deleted 93% of the 
trivial and 95% of the redundant information. ^ 

Performance on .the remaining three rules T^as generally at a level set 
by seventh-tenth graders and considerably less eijfficient than that of the 
four-year college populations. For comparative purposes, the junior 
college performance (collapsed across Summary Type and Stories) is shown in 
Figure 1, together with the comparable data from experts and four^year 
colleg^e students. 

Consider first the superordination rule. ' Tlik conditional probability 
of efficient superordination on the unconstrained summary was .45 for 
junior college subjects compared with .28 for seventh graders and .60 for 
tenth graders in Experiment 2 ( p < ..05). On the constrained summary, the 

^ 

conditional probability of an efficient superordination was .69' for junior 
college students compared with .51 for ^eve^h graders and .82 for tenth 
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graders (p < .05). On this rule the junior college performance fell 
approximately midway between seventh and tenth grade performance levels. 

Junior college "students had particular difficulty dealifig with the 
selection and invention rules. These data are also included in Figure 1. 
Analysis of variance comparing junior college students' selection rule use 
to that of the seventh graders and college students of Experiment 1 
suggested that the junior college students were performing on essentially a 
seventh grade level but significantly worse than foulr-year college 
students, F(l,38) » 16.03, p < .001. The only time junior college students 
appeared to do better than seventh graders was on the unconstrained summary 
(grade x summary type interaction, F(l,38) = 6.95, £ < .05), but^when 
pressed- .for space, both groups performed equallj^ (poorly). 

Junior college students' use of the invention rule was also poor, as 
shown in Figure 1. Again junior college students performed at 
approximately the level set by seventh graders and significantly less well 
than four-year college students, F(l,38) = 20.16,^2 < .001. 

Confirming the global claims of educational psychologists, it would 
appear th&t students from less academically privileged backgrounds perform 
poorly on a variety of text-processing strategies, including summarization. 
These data take us beyond this global claim by providing a more fine 
grained analysis of where the students are experiencing particular 
problems. The ability to delete trivial or redundant material is intact, 
at least with the very simpj^e expository materials used in these studies. 
The strategies needed for adequate manipulation of topic sentence rules 
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are, however, much more problematic for th^e students, 'junior college 
students, even those with no^iagnosed reading or writing problems, perforu 
on a level comparable to that of seventh graders from regular junior high 
schools* 

GENERAL DISCUSSION 
This series ^of studies provide empirical confitmation of the Kintsch 
and van Dijk (1978) theory of prose comprehension. By applying a scoring 
system based on the moiit common macrorules, it was possible to^ capture the 
flavor of much of the data. However, it^ould be noted that the more 
mature summarizers differed from the immature in ways that were not 
captured by the simplified scoring procedure. First, and most obvio^isly^. 
the raters had no difficulty identifying the product"* of the less mature 
writers, and, indeed, it was necessary to instruct them to ignore* style and 
concentrate only on rule use when scoring. Quite simply, college students 
and experts write better as well as use rules more efficiently. Another 
obvious developmental difference was the marked tendency on the part of the 
more mature subjects to rearrange material across paragraphs, combining 
according to comon topic. This was a popular strategy used by experts in 
this study and reported previously as symptomatic of college students . 
(Brovm, Day, & Jones, in press). 

Another subtle condensation manipulation used by the more experienced 
students was the tendency to' capitalize on infetreiitial reasoning. For 
example, one expert^ reported using this ploy ^liberately. "The audience 
should be aware that the gnimals were waiting during the day or sleeping 
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during the day due to the heat, they can makegthat conclusion themselves. 
It Is not necessary to make It explicit that .the animals are, waiting 
because of the heat of the day and that; the desert temperature becomes 
cooler during the night." Subtle writing procedures that rely on the 
readers' Inferential reasoning abilities were not captured by the crude 
scoring procedures used here, and- they certainly deserve future attention.- 

The developmental data extend the Kintsch ancj van Dijk (1978) model 
that Is silent' with respect to the differential difficulty of applyitig the 
macrorules. Throughout this series Vf studies a clear developmental 
pattern was found, with deletion ruxes emerging first followed by 
superordlnatlon and then selection. Invention,' the most difficult rule,* ^ 
was late developing. We believe that the £lve ^ules differ in their ease 
of application because they demand different degrees of text manipblatlon 
on the part of the learner, and perhaps because they depart to a greater or 
lesser extent from the already existing strategy favored by the younger 
participants. This has been called the copy-delete strategy (Brown, 1981; 
Brown, Day, & Jones, in press) because fifth and seventh grade. and -junior 
"college students summarize texts primarily by deleting, or copying ngar 
verbatim the words actually present in the text. 

Consider the five rules of deletion, superordlnatlon and topic 
sentence manipulation in termd of how far they depj^rb from thi copy-delete 
strategy. Obviously, the easy deletion rules map straight onto the 
existing strategy; ui>necessary material is merely deleted. Copy-delete 
works quite well for sup*rordlnatlon with the ;ulnor "departure that the 
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students must add a superordlnate In place of a deleted list. But in order 
^ to use the topic sentence rules appropriately, the students must abandon 
either the sequential unit by unit approach or both the sequential approach 
and the copy-delete principle. To use the selection rule, the students 
must have some realization of the unique status of the topic sentences. 
This would d€»mand disrupting the sequentlallty rule and giving unique 
status to topic sentences, for example, by selecting them first to form the 
scaffolding of the summary \as experts do (see Experiment 2). The main 
feature of the copy-delete rule still applies, however. In that one can 
copy the selected topic sentence straight from the text. 

The Invention rule Is difficult because It departs most radically from 
the favored copy-delete ploy.. Students must now add something of their 
own, a synopsis In their own words of the Implicit meaning of the 
paragraph. The Invention rule, . therefore, requires that the students add 
Information rather than just delete, select or manipulate sentences already 
provided for them. It Is these processes that are the essence of good 
summarization, that are used with facility by experts and that are most 
difficult for novice learners. , > , * , 

On a more speculative note, there is evidence that partially. adequate 
Strategies such as copy-delete are not just way-stations on the road to 
expert strategies; they may actually Impede progress. Copy^-delete is a 
partially adequate strajfelegy in that It results in a product that Is- 
recognizably a summary, an outline, or a set of notes and teflichers will 
accept the product as adequate (Brown, 1981). Berelter and Scardamalla (JLn 
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press) describe another partially adequate writing strategy adoptee} by 
novice writers. A common composition tactic of young writers is to %ell 
all they know on a topic irrespective of the writing assignment. For * 

example, when writing an essay on winter, the child might begin with "I 

J. 

think winter is the best time of year because you can make snowmen"; the 
child will then proceed for many more sentences teUtpg all she knows about 
snowmen. Having exhausted that topic, the child will declare that the 
composition is ended, seemingly having "forgotten" the original purpose of 
the essay. This general ploy is referred to as the knowledge-telling 
strategy (Berfeiter & Scardamalia, in press). 

The knowledge- telling strategy is a device favored by many novice, and 
not so novice, writers. And it bears many similarities to the copy-delete*" 
strategy; like the copy-delete strategy, the knowledge-telling strategy is 
difficult to eradicate because it is partially successful. Knowledge- 
telling results in a recognizable product acceptable to teachers. Writing 
gets done. 

Bereiter and Scardamalia argue that the knowledge-telling strategy 
gives way to reader-based, responsive, mature writing only with great 
difficulty because of . the partial success of the Inadequate strategy. We 
would like to argue that partially adequate strategies such a^ copy-delete 
and knowledge- telling are maintained by inexperienced writers because they 
do result in intermittent reinforcement and are recognizable attempts to 
get the job done. The process of development is not just one of acquiring 
increasingly more refined and sophisticated strategies; development 

2a'. 
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Involves the systematic consolidation of mature strategies, combined with 
the rejection of plauslbj-e but less efficient habits (Brown, Bransford, 
Ferrara, & Camplone, In press) ♦ 

Finally, the studies can be characterized as exercises In applied 
cognitive science, or cognitive engineering (Norman, 1979) ♦ Taking as a 
starting point a basic theory of prose comprehension (KlntsQh & van Dyke, 
1978), we proceeded to map the developmental progression associated with 
the passage from novice to expert. This Information enabled us to go 
beyond the global claim that Immature students exfierience difficulty with 
text processing, Including summarization, and to identify specific 
operations that prove particularly troublesome at certain stages in the 
developmental progression. ' Diagnosis of the exact location of the 
difficulty implicated that rules of selecting, and particularly Inventing, 
topic sentences are particularly difficult for younger children and for 
junior college students, even those with no diagnosed reading and writing 
problems. Even four-year college students perform less well than experts 
on two indices of efficiency, the ability to combine information acroa^ 
paragraphs ,and the ability to provide a, synopsis of a paragraph In the 
absence of an explicitly stated topic sentence. One advantage of these 
more pre^-.se, theory-driven diagnoses is that remediation can be tailored 
to a, student's specific weaknesses. Attempts to devise such student** 
responsive training have proved successful (Brown, Camplone, & Day, 1981; 
Day, 198'0). , ^ \ ' " . 
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Table 1 
Use of Deletion Rules 
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Material 
Summary 

Age 

5th grade 
7th grade 

10th grade 

College 



Trivial ^Units 



Redundant Units 



Unconstrained 

.91 ' 
.85 
.82 
.90 



Constrained 

.95 
.93 
.91 
.95 



Uncons trained Constrained 



.97 
.92 
.92 
.91 



.96 
.95 
.93 
.98 



Table 2 
Use of Superordination Rule 
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.Age^ 



Delete entire sentence 
Repeat entire sentence 
S up er 0 r d i na t i o n : 

Efficient 

Inefficient 

P/Efficient 
given not deleted 



Unconstrained Summary 
5th 7th 10th Coll^^^ e 



60-Word Summaxry 
5th 7th iQth Colleg e 



•57 .27 .19 .22 

.n .33 .23 AO 

.17 .21 .46 .42 

.14 .20 .12 .28 

.44 /28 /.60 .56 



•54 .39 .33 .36 

.10 .20 :06 .03 

.26 .31 .54 .55 

.10 .10 .06 .07 



.52 .51 .82 



.85 
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Table 3 

Use of Selection and Invention Rule 



Rule 



Selection 



Age 

5th grade 
7th grade 
lOth grade 
' College 



Summary Type UnconstTained Constrained 



.29 

.34 
.56 
.72 



.28 
.33 
.52 
.53 



Invencion 



Unconstrained Constrained 



-.14 
.28 
.36 
.52 



.14 
.23 
.38 
.46 
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/Table 4 



Examples^-of Experts' Rule Verbalization 



Deletions 



Superordination 



Topic Sentence Selection 



Topic Sentence Invention 



Combining Across Paragraphs 



'The details are dropped for a summary of this type. 
. You need the generalizations, not the details." 
(trivia) 

p 

"This essay wastes two senfences. Bot^ state the 
simple fact that desert animals are nocturnal due 
to the heat. You can omit one." (redundancy) 

"One thing I've done is drop the kinds of plants. 
Instead of writing daisies, poppies, marigolds and 
lilies, all I've written is 'annual plants', again 
leaving out details and talking about generalization,. 

"This sentence contains the essential point of the 
paragraph ,^ it states the process by which plant life 
is. maintained. It .has to be included in any summary. 

"The paragraph is about the cycle of the annual 
plants that produce seeds, wait until rainfall, 
bloom, prx)duce seeds again, etc. Although it . 
doesn't say so explicitly, all you need is to state 
this cycle then you can drop the rest." 

"In the first two paragraphs the only really essen-- 
tial information is the facts about the heat and 
the lack of water in the desert. I'll combine the' 
first two paragraphs into only two sentences ~ / 
that, contains all the. inf opnation that I need. One 
sentence is simple, the other is a compound sentence. 

"On the third and fourth paragraphs^ information is 
given about plant^ life. The third is about annual 
-flowers and the fourth is about the cactus, a flower 
particular to the desert. ' Now, a lot of information 
is given there. Th€s details can be dropped. And. 
the two paragraphs can be combined to one single 
paragraphs since they both deal with plant life." 
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\^ , FIGURE CAPTION 

Figure 1. Use of the selection, invention and superordinatiou 
) 

rules by college students* of varying degrees of expertise. 
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